[Bugfix][Spec Decode] Scope DSpark backend inheritance to DeepSeek V4 - #52809
Merged
zyongye merged 3 commits intoAug 21, 2026
Merged
Conversation
Co-authored-by: Codex <noreply@openai.com> Signed-off-by: mgoin <mgoin64@gmail.com>
mgoin
marked this pull request as ready for review
August 19, 2026 14:54
Member
Author
|
/ci run |
|
✅ Triggered Buildkite CI #84610 for commit |
Co-authored-by: OpenAI Codex <codex@openai.com> Signed-off-by: mgoin <mgoin64@gmail.com>
Member
Author
|
/ci run |
|
✅ Triggered Buildkite CI #84667 for commit |
zyongye
approved these changes
Aug 21, 2026
zufangzhu
pushed a commit
to zufangzhu/vllm
that referenced
this pull request
Aug 24, 2026
…vllm-project#52809) Signed-off-by: mgoin <mgoin64@gmail.com> Co-authored-by: Codex <noreply@openai.com> Co-authored-by: OpenAI Codex <codex@openai.com> Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
2 of 6 tasks
am-cohere
pushed a commit
to am-cohere/vllm
that referenced
this pull request
Sep 1, 2026
…vllm-project#52809) Signed-off-by: mgoin <mgoin64@gmail.com> Co-authored-by: Codex <noreply@openai.com> Co-authored-by: OpenAI Codex <codex@openai.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
#52288 made an unspecified DSpark draft backend inherit the target's backend.
That fallback is required for DeepSeek V4 because its target and draft layers
share a KV-cache layout, but it is not valid for every DSpark architecture.
For example, a Kimi MLA target can use a Qwen3/SWA DSpark draft. Inheriting
FLASHINFER_MLAmakes draft initialization fail because that backend does notsupport sliding-window attention.
Keep the explicit draft override behavior, but inherit the target backend only
when the normalized DSpark draft has
model_type == "deepseek_v4". OtherDSpark drafts leave the backend unset so their own attention kind participates
in normal backend auto-selection.
Why this is not a duplicate
This is a narrowly scoped follow-up to the regression introduced by #52288.
No open PR matched the backend-inheritance regression. Nearby #48381 addresses
the draft KV-cache dtype and #51042 addresses DSV4 SWA index width; neither
changes draft backend selection.
Tests
The regression test covers DSV4 inheritance, non-DSV4 auto-selection, and an
explicit draft override.
Model validation
On 4x B300, main reproduced the non-DSV4 failure with a Kimi-K3 MLA target and
the Qwen3/SWA
RedHatAI/Kimi-K3-speculator.dspark: the inheritedFLASHINFER_MLAbackend was rejected because sliding-window attention is notsupported. With the independent-selection behavior restored, the target chose
FLASHINFER_MLA, the draft choseFLASHINFER, the engine initialized, and a32-token chat request completed.
No output-accuracy evaluation was run because this changes startup-time backend
routing only. The DSV4 path retains the behavior validated in #52288.
AI assistance
This PR was produced with AI assistance. It is intentionally a draft; the human
submitter must review every changed line before marking it ready.